Papers with few-shot learning capabilities
VIST5: An Adaptive, Retrieval-Augmented Language Model for Visualization-oriented Dialog (2023.emnlp-demo)
Copied to clipboard
| Challenge: | a number of visualization-oriented dialog systems rely on tailored dialog agents to analyze domain-specific data and operate domain- specific visualization tools and libraries. |
| Approach: | They propose a visualization-oriented dialog system that focuses on easy adaptability to an application domain and easy transferability of language-controllable visualization library functions between applications. |
| Outcome: | The proposed system is based on a retrieval-augmented T5 language model that leverages few-shot learning capabilities to enable a rapid adaptation of the system. |
PromptRefine: Enhancing Few-Shot Performance on Low-Resource Indic Languages with Example Selection from related Example Banks (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated impressive few-shot learning capabilities through in-context learning. |
| Approach: | They propose a novel Alternating Minimization approach for example selection that improves ICL performance on low-resource Indic languages. |
| Outcome: | The proposed approach outperforms existing frameworks for retrieving examples on low-resource Indic languages. |
Multi-Scale Prompt Memory-Augmented Model for Black-Box Scenarios (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for few-shot text classification require numerous LMs’ calls to search optimal prompts, thus resulting in overfitting performance and increasing computational cost. |
| Approach: | They propose a multi-scale knowledge prompt-based memory model that extracts instance-level and class-level knowledge and stores them in memory banks during training. |
| Outcome: | Experiments on different benchmarks and parameter analysis demonstrate the effectiveness and efficiency of MuSKPrompt in black-box few-shot text classification tasks. |
CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on counterfactual data augmentation only considered restricted classes of perturbations, limiting their effectiveness. |
| Approach: | They propose a retrieval-augmented framework for creating diverse counterfactual perturbations for CDA. |
| Outcome: | Experiments on natural language inference and sentiment analysis show that the proposed framework can be used to encourage diversity in manually authored perturbations. |
On the Multilingual Capabilities of Very Large-Scale English Language Models (2022.lrec-1)
Copied to clipboard
| Challenge: | Generative Pre-trained Transformers (GPTs) have been scaled to unprecedented sizes in the history of machine learning. |
| Approach: | They investigate the potential and limits of Generative Pre-trained Transformers in three tasks . they find it can be almost as useful for many languages as it is for English . |
| Outcome: | The proposed model can perform tasks in five different languages, and its potential is explored . it can learn from a few examples "via text interaction" and is scalable to many languages . |
Learning to Search Effective Example Sequences for In-Context Learning (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods address these factors in isolation, overlooking their interdependencies. Existing approaches focus on sequence selection, while focusing on the sequence of examples. |
| Approach: | They propose a method that considers key factors involved in sequence selection and incrementally builds the sequence. |
| Outcome: | Experiments across various datasets and language models show that the proposed method significantly reduces the search space and improves performance. |
Structural Supervision Improves Few-Shot Learning and Syntactic Generalization in Neural Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have not investigated the relationship between a token's frequency in the training corpus and syntactic properties models learn about it. |
| Approach: | They develop controlled experiments that probe models’ syntactic nominal number and verbal argument structure generalizations for tokens seen as few as two times during training. |
| Outcome: | The proposed models can make syntactic generalizations for tokens seen as few as two times during training and transfer them to transformed contexts. |
Designing Informative Metrics for Few-Shot Example Selection (2024.findings-acl)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) have shown remarkable few-shot learning capabilities when provided with properly formatted examples. |
| Approach: | They propose a complexity-based prompt selection approach for sequence tagging tasks that uses certain metrics to align the syntactico-semantic complexity of test sentences and examples. |
| Outcome: | The proposed approach achieves state-of-the-art performance on few-shot NER, with 5% improvement in F1 score. |
Reasoning Graph Enhanced Exemplars Retrieval for In-Context Learning (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods focus on semantic similarity between queries and candidate exemplars, while logical connections between reasoning steps can be beneficial to depict problem-solving process. |
| Approach: | They propose a method to retrieve exemplars with semantic and structural similarity using a graph kernel. |
| Outcome: | The proposed method is superior to state-of-the-art retrieval-based approaches on mathematics and logical reasoning tasks. |
Retrieval-Augmented Generation for Large Language Model based Few-shot Chinese Spell Checking (2025.coling-main)
Copied to clipboard
| Challenge: | Existing LLM-based Chinese spelling check methods rely on fixed prompt samples . existing methods are limited by technical bottlenecks, complex recognition environments, and individual differences . |
| Approach: | They propose a framework called RagID to provide well-chosen prompt samples . they propose to use semantic-based similarity search and iterative discriminator mechanism . |
| Outcome: | The proposed framework can provide well-chosen prompt samples and reduce overcorrection issues in Chinese spelling check tasks. |
The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Language models with less than 100B parameters perform poorly on chain-of-thought reasoning . we aim to equip smaller LMs with the step-by-step reasoning capability . |
| Approach: | They propose to equip smaller LMs with the step-by-step reasoning capability by tuning with CoT rationales. |
| Outcome: | The proposed dataset outperforms large LMs on 4 domain-specific tasks even with demonstrations . |
What Makes a Good Order of Examples in In-Context Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) demonstrate impressive few-shot learning capabilities via in-context learning (ICL). |
| Approach: | They propose to use unlabeled data to evaluate order performance . they propose to filter out subsets of orders with label fairness and select the most influential order for each test instance. |
| Outcome: | The proposed method is superior over strong baselines and validates generalizability across settings. |
SERE: Structural Example Retrieval for Enhancing LLMs in Event Causality Identification (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated strong performance across various NLP tasks, but their effectiveness in ECI remains limited due to biases in causal reasoning. |
| Approach: | They propose a structural example retrieval framework that leverages LLMs’ few-shot learning capabilities to help LLM models in ECI. |
| Outcome: | The proposed framework leverages LLMs’ few-shot learning capabilities to guide LLM models in causal reasoning, mitigating bias and improving accuracy. |